Indexical Grounding for a Mobile Robot

We have outfitted a mobile research robot with several sensors and algorithms designed to facilitate smalland large-scale navigation and natural language interaction. We begin with a parser using a large, hand-crafted English grammar and lexicon. We then add a standard gradient navigation algorithm for local obstacle avoidance, and a line segment comparison algorithm for basic, high-performance location recognition. The result is a full end-to-end system for natural-language-driven, mobile robotics research. The theme of grounding-mapping linguistic references to the corresponding real-world entities-runs throughout our approach. After the parser simplifies linguistic symbols and structures, we must connect them to the basic concepts that they represent, and then to our system's specific sensor readings and motor commands, to make natural language interaction possible. Additionally, many of the symbols we must ground are indexicals with critical contextual dependencies. We must therefore handle the implicit context that spatial communication carries with it. Thesis Supervisor: Deb Kumar Roy Title: Associate Professor of Media Arts and Sciences